A component-model approach to determining what to learn*

Research in machine learning has typically addressed the problem of how and when to learn, and ignored the problem of formulating learning tasks in the first place. This paper addresses this issue in the context of the CASTLE system,1 that dynamically formulates learning tasks for a given situation. Our approach utilizes an explicit model of the decision-making process to pinpoint which system component should be improved. CASTLE can then focus the learning process on the issues involved in improving the performance of the particular component. 1. Determining what to learn A theory of learning must ultimately address three issues: when to learn, what to learn, and how to learn. The overwhelming majority of research in machine learning has been concerned exclusively with the last of these questions, how to learn. This work ranges from work in purely inductive category formation to more knowledge-based approaches. The aim of this work has generally been to develop and explore algorithms for generalizing or specializing category definitions. The nature of the categories being defined---i.e., what is being learned---is rarely a consideration in the development of these algorithms. For purely inductive approaches, this is entirely a matter of the empirical data that serves as input to the learner. In explanation-based approaches (EBL), it is a matter of the user-defined ``goal concept''---in other words, input of another sort. In neither case is the question of what is being learned taken to be within the purview of the model under development.2 Some work---in particular that in which learning has been addressed within the context of performing a task---has addressed the first question above, namely, when to learn. A * The research presented in this paper was carried out at the Institute for the Learning Sciences at Northwestern University, and is discussed in full in the author’s Ph.D. thesis [Krulwich, 1993]. 1CASTLE stands for C oncocting A bstract S trategies T hrough L earning from E xpectation-failures. 2 Although the need to address this question in the actual deployment of EBL algorithms has been explicitly recognized [Mitchell et. al., 1986, p. 72]. common approach to this issue, known as failure-driven learning, is based on the idea that a system should learn in response to performance failures. The direct connection this establishes between learning and task performance has made this approach among the most widespread in learning to plan. For the most part, however, even these models do not address the second question above, what to learn. In many cases, this is because the models are only capable of learning one type of lesson. What to learn is thus predetermined. For example, many systems that learn exclusively from planner success always learn the same thing, namely a generalized form of the plan that was created (e.g., [Mitchell, 1990]). Similarly, many systems that learn from plan outcomes always learn the same type of planning knowledge--e.g., when it is feasible to make certain simplifying assumptions or to defer planning---from each situation (e.g., [Chien, 1990; DeJong et. al., 1993]). Even systems that can learn more than one thing generally do so in a predetermined and inflexible fashion (e.g., [Minton, 1988; Hammond, 1989]). While this type of solution can often be effective for any individual application setting, it fails to provide an account of how a learning system could determine for itself what to learn, and do so in a manner that is flexible enough to take account of the internal and external context of learning. For a system that is capable of learning a wide variety of types of concepts, in a wide variety of settings, the number of hardwired mapping rules required to do this would be very large, and the rules themselves would get very difficult to manage or reason about, and may even be impossible to formulate. More importantly, however, is the fact that hard-wired rules of this type do not provide a theory of determining what to learn. Just as a set of rules for actions can result in intelligent behavior without providing a foundation for the actions, hard-wired rules for determining what to learn can be effective but nonetheless do not necessarily provide a theory underlying these decisions. The point is that just as complex decisions about actions are very difficult to formulate using hard-wired rules, and thus require inference, so too complex decisions about what to learn require inference. 2. An everyday example Consider the case of a person cooking rice pilaf for the first time. The last step in the directions says to ``cover the pot From: AAAI Technical Report SS-94-02. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved. and cook for 25-30 minutes.'' Suppose the person starts the rice cooking and then goes off to do something else---say, clean up the house. In the interim, the pot boils over. When the person returns to the kitchen a half-hour later, the rice pilaf is ruined. What should be learned from this sequence of events? Intuitively we can imagine a number of lessons that might be learned: • Whenever a covered pot containing liquid is on the stove, keep an ear peeled for the sound of the lid bouncing or the sound of the water bubbling. • Do not put a covered pot with liquid in it over a high flame, because it will boil over. The flame should be turned down. • When cooking over a high flame, leave the pot uncovered or the lid ajar. • Don't do loud things while cooking on the stove. • When cooking liquid in a covered pot, stay in the kitchen, because it's hard to hear a pot boiling over from the other rooms. • Don't cook over high flame when busy. While all of these lessons are sensible, they are very disparate, in that they address very different issues. The lessons concern different aspects of behavior, refer to different portions of the agent's plan, and are expressed in different vocabularies. It is difficult to imagine how any learning process that did not distinguish among these alternatives in some way would be capable of such diverse behavior. Rather, it seems more likely that before the agent can undertake the task of learning from the mistake, he must select a lesson (or set of lessons) to learn. In other words, given that the agent has decided to learn from the mistake, and given that he is capable of carrying out the learning task, he still has to first determine what to learn. We see, then, that the agent could learn several things in response to the rice pilaf boiling over. Which of the lessons the agent should learn, whether changes to the cooking methods, the idea of staying in the kitchen, or of tuning his perceptual apparatus, depend on the agent's perceptual and planning abilities, and on his knowledge of the domain. The key point is that many different lessons are possible. Any approach to determining what to learn must be flexible enough to account for this diversity. 3. Modeling cognitive tasks What would an appropriate theory of determining what to learn look like? Imagine the thought processes going on in the agent's head (consciously or subconsciously) in viewing his situation and considering what lesson to learn: • Why was the rice ruined? The rice boiled over. • Could I have done something differently at the time I started the rice cooking to prevent the problem? Yes, I could have lowered the flame or uncovered the pot. • Without doing this, could I have prevented it from boiling over? Yes, if I had heard it. • Could I have heard it boiling over? Maybe I could have, if I'd paid more attention. • Why couldn't I hear it boiling over? I was using the vacuum in the living room. • Could I have planned things differently to enable me to hear? Yes, I could have delayed vacuuming or stopped every few minutes to check the rice.